19 Approximate Inference

• v: visible variables
• h: hidden variables

Challenge of inference: compute p(h|v) or take expectation with respect to it.

Review:

Direct interaction in undirected model / explaining away interaction between mutual ancestors of the same visible unit in directed model ==> interaction between latent variables => Intractable inference problem.

• 19.1 Inference as Optimization
• 19.2 Expectation Maximization
• 19.3 MAP Inference and Sparse Coding
• 19.4 Variational Inference and Learning
• 19.5 Learned Approximate Inference
• Paper: Auto-Encoding Variational Bayes